Pipeline, March/April 1996, vol.7, no.2
Copyright © 1996 Silicon Graphics

The Basics of File Locking

There are two primary types of locking that are discussed in this article: shared and exclusive. Shared locking allows multiple locks on a given file, while exclusive locks require sole access to the file specified. For example, if an application simply reads and writes data to a file, a shared lock is used when reading data, and an exclusive lock is used when writing data. Using an exclusive lock when writing data keeps two or more users from overwriting information that each of them places into the file. A shared lock is used when the application is only looking at the file to handle situations where someone holding an exclusive lock on a file changes the data. In that case, the application may want to read whatever exists in the file after the changes are made to the file.

In addition to shared and exclusive locks, there are also waiting and non-waiting states. An application can either wait until a file lock can be accessed, or return immediately so that the application can complete other tasks. Simple locking programs will wait until they have access to the file and make changes to the file when it is their turn. More sophisticated programs can test and determine if a lock can be set, and if not, perform other operations and try to lock again later. It is up to the programmer to determine how best to utilize shared and exclusive locks in combination with waiting and non-waiting states.

In Figure 1, two programs, program #1 and program #2 are trying to exclusively lock the same text file. Program #1 receives the exclusive lock first, and program #2 waits for program #1 to release the lock the file. As soon as program #1 releases the lock, program #2 can obtain the lock. Note that if program #1 and program #2 were trying to get shared locks on the text file, both could lock the file at the same time.

<not available in ASCII>

Figure 1. Exclusive locking

System Locking Functions

Many Unix systems have a number of locking functions that go through the system kernel and implement low level locking specific to the system. Each of the functions listed below are provided on SGI systems to allow programmers to implement locking in their applications.

Flock

Flock(3B) provides advisory locks on files, allowing either shared or exclusive locking on a file. Advisory locks are locks that provide locking capabilities on a file, but do not exclude other processes that do not use locking from accessing the file. Note that this can lead to data inconsistencies, but as long as all processes accessing a specific file attempt to acquire a lock first, this should not be a problem.

In addition to advisory locking, flock also provides a non-blocking locking hook, which is the same as a non-waiting state. By default, flock will wait until the lock becomes available instead of returning an error condition.

Lockf

Lockf(3C) provides both advisory and mandatory locks on files. Mandatory locking means that processes wanting access to a file cannot acquire it until the current owner of the lock on the file gives the lock up. This type of locking can be used by setting certain permission bits on a file; refer to the manual page for chmod(1) for more information on which permission bits to set.

Along with generic file locking, lockf also allows byte ranges to be locked in a file. This is useful when multiple processes are actively trying to read and write information to the same file. By locking a section of a file instead of locking the entire file, multiple processes can update information in different sections of a file at the same time.

Fcntl

Fcntl(2) and fcntl(5) provides the same features that flock and lockf have, but through a different interface. With fcntl, different flags are passed to indicate the type of locking, whether a byte range has been specified, and whether a waiting or non-waiting state should be used. F_GETLK is used to get lock information, and F_SETLK is used to attempt a lock. F_SETLKW can be used instead of F_SETLK in order to indicate that the process should sleep waiting to acquire the file lock.

Programming Examples

There are three programming examples included at the end of this article in order to help a programmer become more familiar with how to use file locking. Each example takes advantage of a different lock routine in a different fashion. To try out each piece of code, the user should open multiple login windows on the system.

Example 1

This example uses flock to set an exclusive lock on a file. The code will open the filename specified on the command line in read-only mode, and try to set the lock on the file. Once the lock is in place, the user can press <ENTER> in order to unlock the file and exit. If the lock is already set by a different process, the program will wait until the lock is released before getting the lock. If there is an error, you will see the message:

Could not set lock. Lock is currently held.

This example is more sophisticated than the first example. This program opens a file for updating with fopen(3S) instead of open(2), and tries to lock the file with fcntl. In order for this program to work, the file specified on the command line should contain a number in it (e.g., 0). The number will be read from the file, incremented by one, and then written back to the file. If the lock is already set on the file by someone else updating the number, the following message will be printed:

Could not set lock. Lock is currently held.

Example 3

This program allows the user to try and lock byte ranges in a file. On the command line, the user specifies a filename, along with a range of bytes within the file. After the file is opened, the range of the low and high values are verified, and then the byte range is locked with lockf. If the range specified is already locked by another process, the following message will be printed:

Could not set lock. Lock is currently held.

Remote NFS File Locking

Most application developers today are required to write their programs so that they work across NFS mounted filesystems. On SGI systems (as well as many other Unix systems), careful attention needs to be paid when writing locking code so as to avoid wasting development time on locking issues with NFS.

When a program on an NFS client tries to lock a file, a number of things happen. First, the kernel on the NFS client receives the lock request, and determines where the file actually resides. If the file is found on a NFS mount point, the kernel sends the lock request to a program called rpc.lockd (refer to the manual page for lockd(1M) for more information) running on the NFS client. The rpc.lockd application is responsible for receiving and transmitting lock requests back and forth between NFS client and server. Once the rpc.lockd process on the NFS client receives the request from the kernel, it sends the request to the rpc.lockd process running on the NFS server. On the server, rpc.lockd requests the lock on the local NFS server system. If the lock is successful, rpc.lockd on the NFS server tells the rpc.lockd on the NFS client that the request was successful. At that point, rpc.lockd on the NFS client tells the kernel that the lock was successful, and asks rpc.statd (statd(1M)) to monitor the lock request. Refer to Figure 2 for a graphical display of what happens with NFS lock requests.

<not available in ASCII>

Figure 2. Flow of NFS lock requests

The benefit of having NFS clients transmit the lock request to the NFS server is that all lock requests will then be handled on one system in the proper order. If rpc.lockd is not running on both the NFS client and server systems, the lock request will not be handled properly, and the application cannot be sure that there is a valid lock on the file.

Some programmers trying to avoid locking their files altogether and simply implement a data synchronization strategy. Unfortunately, this method does not always work correctly when running across NFS. Refer to the section titled "Synchronization and Locking Across NFS" for a discussion of the dangers of data synchronization as opposed to file locking.

Ensure that rpc.lockd is Running

There is a chkconfig(1M) option called lockd that is available on IRIX systems. First, check and see if the lockd configuration flag is turned on by running the command:

% /etc/chkconfig | grep lockd

# /etc/chkconfig lockd on

If lockf, flock, or fcntl is used in the application along with rpc.lockd on the NFS client and server, the program will work the same as if it were run on a local filesystem.

In the previous section, three programming examples were used to illustrate record locking on a local system. If these programs are run on NFS clients on files that are on a NFS mounted filesystem, the same behavior should be seen.

Synchronization and Locking Across NFS

One of the questions asked occasionally deals with data synchronization and locking on remote filesystems. On SGI systems, there is a difference between the two techniques.

Programmers can use data synchronization by setting certain flags on the file descriptors in their programs in order to specify that the data should be written to the server before the kernel returns to the program from the function call. This is supposed to ensure that all reads and writes to the files will complete before any other read or write can occur. There are some problems using this technique across remote filesystems, however. Because of this, data sychronization should not be used as a substitute for file locking.

SGI NFS clients (along with some other NFS implementations) use a local buffer cache, where data written to a server is stored on the client in a cache that can be directly accessed without having to go to the server to re-read the data. The benefits are significant; most reads can go to the local cache on the NFS client instead of having to read the data directly from the NFS server.

However, a potential problem exists when a file can be updated multiple times within the length of time specified by the filesystem's modification time granularity. For example, on SGI systems, the granularity of the modification time for files stored on disk for an EFS or XFS filesystem is one second. If a file were modified twice within a single second, an NFS client might not notice this and read earlier data from its buffer cache, incorrectly assuming that its local cache is as up to date as the file on the server. Refer to Figure 3 for a more detailed look at how this problem can occur.

<not available in ASCII>

Figure 3. A common problem in using data synchronization

The only way to avoid this problem is to use one of the locking functions (lockf, flock, or fcntl) instead of trying to synchronize the information on each write. The locking routines on an NFS client will avoid writing to the buffer cache, and instead write directly to the server. Note that there are some trade-offs here. It might not be appropriate to lock all read(2) and write(2) operations, since that will decrease NFS performance (because the program is not taking advantage of the local buffer cache). However, if locking is not used in critical regions of the program, there is a risk that data may be lost. The best way to handle this is to be conservative and lock only when it is absolutely necessary, and perform normal reads and writes in every other case.

One other point needs to be made here for completeness. Developers sometimes want to be sure that the data they write out across an NFS mount goes to the server and onto the remote disk before the write call returns. In order to do this, the programmer must use the open call with the O_SYNC flag, or use the function fsync(2) with all write operations. In addition, the developer must also specify the wsync option on the exported filesystem on the server. If this is done, a write will not return until the buffer has been written across the NFS mount and to the disk. Most NFS mounted filesystems are not exported with the wsync option. Regardless of whether file locking is used, the filesystem must be exported on the server with the wsync option if the developer wants to ensure that the data is written before the write returns. For more information about wsync, refer to the exports(4) manual page.

---

When using NFS3 instead of NFS, the wsync option has no effect. This is because all write operations on an NFS3 filesystem will occur synchronously to the NFS3 server making the wsync option redundant.

---

Conclusion

Whether using lockf, flock, or fcntl, file locking is an important and useful feature for maintaining data integrity on files that are accessed by multiple programs from multiple systems. Although file locking might seem difficult, it is actually quite easy to implement and use on both local and remote filesystems. The real issues occur when programmers try to work around the locking mechanisms already provided.

References

For more information on file and record locking, refer to chapter 4, "File and Record Locking", in the Topics in IRIX Programming manual available on-line with InSight(1).

Programming Examples

---

The following programs are broken out
as C source along with a Makefile in
toolbox/src/exampleCode/irix/fileLocking.

---

Example 1

This example uses flock to set an exclusive lock on a file.

/*
 * test1.c: Application to show locking with
 * flock().
 * To compile: cc -o test1 test1.c
 * To run: test1 <filename>
 */ 

#include <sys/types.h> 
#include <sys/file.h> 
#include <stdio.h> 
#include <errno.h>

extern int errno;

int main(int argc, char **argv) 

{ 
	int fd, c;
	/* check the number of arguments specified */ 
	if (argc != 2) { 
		fprintf(stderr, "Usage: %s <filename>\n", argv[0]); 
		return(1); 
	}

	/*	open the file specified on the command line */ 

	if ((fd = open(argv[1], O_RDONLY)) < 0) { 
		perror("Could not open file specified"); 
		return(1); 
	}

	printf("\nAttempting to set a lock on file '%s' ...\n", argv[1]);

	/* 	attempt to set an exclusive lock on the file opened */ 

	if (flock(fd, LOCK_EX) < 0) { 
		fprintf(stderr, "Could not set lock. Lock is currently held.\n"); 
		close(fd); 
		return(1); 
	}

	printf("Lock on file '%s' set.\n", argv[1]); 
	printf("Press <ENTER> to continue -> "); 
	c = getc(stdin);
	printf("\nAttempting to unlock file '%s' ...\n", argv[1]);

	/*	attempt to unlock the file opened */ 

	if (flock(fd, LOCK_UN) < 0) { 
		perror("Could not release lock"); 
		close(fd); 
		return(1); 

	}

	printf("Unlock on file '%s' complete.\n", argv[1]);

	/* close the file and exit */ 
	close(fd); 
	return(0); 

}

---

The following programs are broken out
as C source along with a Makefile in
toolbox/src/exampleCode/irix/fileLocking.

---

Example 2

This example opens a file for updating with fopen(3S) instead of open(2), and tries to lock the file with fcntl.

/*
 * test2.c -- Application to show locking with
 * fcntl().
 * To compile: cc -o test2 test2.c
 * To run: test2 <filename>
 */ 

#include <unistd.h> 
#include <fcntl.h> 
#include <stdio.h> 
#include <errno.h>

extern int errno;

int main(int argc, char **argv) 

{ 
	FILE *fp; 
	int num = 0; 
	struct flock fl;

	/*	check the number of arguments specified */ 

	if (argc != 2) { 
		fprintf(stderr, "Usage: %s <filename>\n", argv[0]); 
		return(1); }

	/*	open the file specified on the command line */ 

	if ((fp = fopen(argv[1], "r+")) == (FILE *)NULL) { 
		perror("fopen() failed"); 
		return(1); 
	}

	/*	zero out all fields in lock structure */ 

	bzero(&fl, sizeof(struct flock));

	/*	set the type of lock to a writer's lock,
	 	and try to lock the file 
	*/ 

	fl.l_type = F_WRLCK; 
	if (fcntl(fileno(fp), F_SETLK, &fl) < 0) { 
		fprintf(stderr, "Could not set lock. Lock is currently held.\n"); 
		fclose(fp); 
		return(1); 
	}

	/* try to read in a number from the file */ 
	if (fscanf(fp, "%d", &num) < 0) { 
		perror("fscanf() of number failed"); 
		fclose(fp); 
		return(1); 
	}

	/*	increment the number read in */ 
	num++;

	/*	rewind our file pointer to the beginning 
		of the file after fscanf() */ 

	rewind(fp);

	/*	try to write a number to the file */ 

	if (fprintf(fp, "%d", num) < 0) { 
		perror("fprintf() of number failed"); 
		fclose(fp); 
		return(1); 
	}

	printf("Lock set [%d].\nPress <ENTER> to continue -> ", num); 
	num = getc(stdin);

	/*	unlock the file */ 

	fl.l_type = F_UNLCK; 
	if (fcntl(fileno(fp), F_SETLK, &fl) < 0) { 
		perror("fcntl() remove lock failed"); 
		fclose(fp); 
		return(1); 
	}

	printf("Lock unset.\n");

	/*	close the file and exit */ 

	fclose(fp); 
	return(0); 
}

---

The following program is broken out
as C source along with a Makefile in
toolbox/src/exampleCode/irix/fileLocking.

---

Example 3

This program allows the user to try and lock byte ranges in a file.

/*
 * test3.c -- Application to show 
 * byte locking with lockf().
 * To compile: cc -o test3 test3.c 
 * To run:
 * test3 <filename> <low byte #> <high byte #>
 */ 

#include <sys/types.h> 
#include <sys/file.h> 
#include <unistd.h> 
#include <stdio.h> 
#include <errno.h>

extern int errno;

int main(int argc, char **argv) 

{ 

	int fd, low = 0, high = 0, c;

	/*	check the number of arguments specified */ 

	if (argc != 4) { 
		fprintf(stderr, "Usage: %s <filename> <low> <high>\n", argv[0]); 
		return(1); 
	}

	/*	open the file specified on the command line */ 

	if ((fd = open(argv[1], O_RDWR)) < 0) { 
		perror("open() failed"); 
		return(1); 
	}

	/*	convert the range arguments specified on 
		the command line to numbers */ 

	low = atoi(argv[2]); 
	high = atoi(argv[3]);

	/*	verify the ranges of numbers */ 

	if (high <= 0) { 
		fprintf(stderr, "Error: high field must be > 0.\n"); 
		close(fd); 
		return(1); 
	}

	if (low < 0) { 
		fprintf(stderr, "Error: low field must be > 0.\n"); 
		close(fd); 
		return(1); 
	}

	if (high <= low) { 
		fprintf(stderr, "Error: high field must be > low field\n"); 
		close(fd); 
		return(1); 
	}

	/*	seek to the low range address given */ 

	if (lseek(fd, low, SEEK_SET) < 0) { 
		perror("lseek() with low value failed"); 
		close(fd); 
		return(1); 
	}

	/*	try to lock the range from low .. high */ 

	if (lockf(fd, F_TLOCK, high - low + 1) < 0) { 
		fprintf(stderr, "Could not set lock. Lock is currently held.\n"); 
		close(fd); 
		return(1); 
	}

	printf("Lock of bytes %d to %d complete.\n", low, high); 
	printf("Press <ENTER> to continue -> "); 
	c = getc(stdin);

	/*	seek to the low range address given */ 

	if (lseek(fd, low, SEEK_SET) < 0) { 
		perror("lseek() with low value failed"); 
		close(fd); 
		return(1); 
	}

	/* try to unlock the range from low .. high */ 

	if (lockf(fd, F_ULOCK, high - low + 1) < 0) { 
		perror("lockf() to unlock file bytes failed"); 
		close(fd); 
		return(1); 
	}

	printf("File range is now unlocked.\n");

	/* close the file and exit */ 

	close(fd); 
	return(0); 
}